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Multiple-choice questions are widely used in higher education and have some important advantages 
over constructed-response test questions. It seems, however, that many teachers underestimate the 
value of multiple-choice questions, believing them to be useful only for assessing how well students 
can memorize information, but not for assessing higher-order cognitive skills. Several strategies are 
presented for generating multiple-choice questions that can effectively assess students ability to under¬ 
stand, apply, analyze, and evaluate information. 


Introduction 

ultiple-choice testing is one of the most com¬ 
monly used forms of assessment in college 
and university settings. Compared to constructed- 
response test questions, such as essays, multiple- 
choice questions enjoy several important practical 
advantages. For instance, for a test of a given length, 
well-chosen multiple-choice questions can provide a 
broader coverage of course content than constructed- 
response questions, and moreover, their scoring is 
generally more statistically reliable. In addition, with 
large classes, the time and effort required to grade 
students’ written answers can be enormous. In con¬ 
trast, the grading of multiple-choice tests is far easier, 
especially when students have used a response sheet 
that is scored using an optical scanner. 


Despite these substantial advantages, many 
teachers tend to have quite a low opinion of mul¬ 
tiple-choice testing. One major reason for this was 
brought to me in full force several years ago during 
a discussion I had with Jenny, an award-winning sci¬ 
ence teacher. As so often happens when two teachers 
get together, the conversation soon turned to work- 
related matters - favourite topics to teach, how to get 
students to read the assigned readings, strategies for 
generating discussion in the classroom. At one point 
in the conversation, I mentioned my then new-found 
interest in multiple-choice testing and was about to 
tell Jenny about a research project that I was just get¬ 
ting off the ground. However, before I could even 
get started, she cut me off at the knees. “Multiple- 
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choice? Are you serious? I would never dream of us¬ 
ing multiple-choice on my tests! All they are good for 
is finding out how well students can memorize, and 
who cares about that?” 

At the time, I was quite surprised by Jenny’s 
strong reaction to my mention of multiple-choice 
testing, but since then I have come to realize that a 
great many teachers are much like Jenny - that is, 
they believe that multiple-choice tests are only good 
for assessing students’ ability to memorize, and that 
they are not useful for assessing higher-order cog¬ 
nitive skills. Research that my students and I have 
recently carried out at Brock University suggests 
that this perception is actually quite widespread. In 
a broad survey of teachers at Brock, we found that 
about 90% believe that multiple-choice questions 
are an effective way to assess students’ ability to re¬ 
member information. However, far fewer of them 
believe that multiple-choice questions can effectively 
assess students’ ability to understand (46%), apply 
(23%), analyze (59%), and evaluate (16%) infor¬ 
mation (Anderson & Krathwohl, 2001). Thus, the 
belief seems to be quite common among university 
teachers that multiple-choice questions are primar¬ 
ily useful for assessing students’ ability to memorize, 
but not much else. 

Although I would certainly agree that mul¬ 
tiple-choice questions are an extremely good way 
to test students’ memory, I would also argue that 
they can do much more than this. Like a number 
of others who have thoughtfully considered this is¬ 
sue (Haladyna, 1999; Linn & Gronlund, 1995), I 
believe that multiple-choice questions can indeed 
be used to assess the higher-level cognitive pro¬ 
cesses mentioned above - understanding, applica¬ 
tion, analysis, and evaluation. For the past several 
years, I have been presenting workshops at colleges 
and universities across Canada in which I familiar¬ 
ize teachers with strategies for generating multiple- 
choice questions that will do more than test stu¬ 
dents’ ability to remember information. One such 
strategy involves taking an existing multiple-choice 
question that tests memory and “ramping it up” to 
a higher cognitive level. I will provide here just one 
example of how this might be done. Consider the 
following question that might be found in the mul¬ 


tiple-choice test bank accompanying an introduc¬ 
tory psychology textbook: 

Question 1. In classical conditioning, 
what name is give to a stimulus that elicits 
a particular response even in the absence of 
any prior training? 

A. conditioned stimulus 

B. unconditioned stimulus* 

C. activational stimulus 

D. discriminative stimulus 

This question asks students to do nothing more than 
remember a fact - namely, that an unconditioned 
stimulus elicits a particular response without any 
prior training. Note that to answer this question cor¬ 
rectly, students do not have to know anything at all 
about the details of classical conditioning or how it 
works; by doing nothing more than remembering 
the definition for unconditioned stimulus, they can 
answer Question 1 correctly. Indeed, students who 
cram the night before the test might very well memo¬ 
rize the definition for unconditioned stimulus, an¬ 
swer Question 1 correctly, and then promptly forget 
the information immediately afterward. 

Unfortunately, low-level items like Ques¬ 
tion 1 are often encountered in test banks, but the 
good news is that we can use this item as a starting 
point to generate a question that requires students 
to have learned something important about classical 
conditioning. So how might we ramp this “remem¬ 
ber” question up to a higher cognitive level? One way 
would be to require students to demonstrate some 
understanding of what an unconditioned stimulus is 
in the context of a situation that they have not previ¬ 
ously encountered, like this: 

Question 1-Revised. Right after a rat 
smells menthol, it is always given Drug 
X, which reliably induces substantial wa¬ 
ter intake. Eventually, the rat drinks water 
whenever it smells menthol, even when it is 
not injected with Drug X. In this situation, 
what is the role of Drug X? 
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A. conditioned stimulus 

B. unconditioned stimulus* 

C. activational stimulus 

D. discriminative stimulus 

Notice that the four alternatives that are provided 
have not changed at all, and the correct answer is still 
alternative B. However, Question 1-Revised requires 
students to have an understanding of the concepts 
underlying classical conditioning, and furthermore 
to be able to classify the role of Drug X in this re¬ 
search setting that they are seeing for the first time. 
With Question 1 -Revised, students who have simply 
memorized the definition for unconditioned stimu¬ 
lus will be at a loss, while students who have truly 
learned about classical conditioning and understand 
how it works will likely be able to answer this ques¬ 
tion correctly. 

It is also possible to generate questions that 
assess higher-level cognitive processes by requiring 
students to read material as part of their course work 
and then asking multiple-choice questions that de¬ 
pend on their having determined the author’s point 
of view, bias, values, or intent. To illustrate, let’s con¬ 
sider a question that I have recently used in a psy¬ 
chology course that I teach. In the course, students 
read How to Think Straight About Psychology, an out¬ 
standing book by Keith Stanovich (2007) that pro¬ 
vides a thoughtful examination of critical issues in 
the sciences generally and in psychology in particular. 
I provide students with study questions that serve as a 
guide as they do their reading, and in class we touch 
upon many of the issues that are dealt with in the 
book. Question 2 is typical of the type of question 
that I would ask students on a test. Note that, as in 
the previous example, students have not encountered 
the particular situation that is presented in the ques¬ 
tion. 

Question 2. According to census data, 
people are having fewer children nowadays 
than they did 50 years ago. Your friend 
Anne tells you that she does not believe this 
because the young couple who live next 
door to her are both under 30 and already 


have four children. If Keith Stanovich were 
told about this, what might you reasonably 
expect him to say? 

A. The census data must be wrong. 

B. Anne’s comment illustrates valid proba¬ 
bilistic reasoning. 

C. Anne’s comment illustrates the use of 
“person-who” statistics.* 

D. The young couple provide an exception 
that actually serves to prove the rule. 

To answer this question, students must have read and 
understood Stanovich’s discussion of what he calls 
“person-who” statistics, which involve the (mis)use 
of data from a single case to argue against some well- 
founded statistical conclusion (“How can you say the 
odds of winning are against you at the casino? I know 
a person who went to the casino and won $10,000!”). 
Note also that Question 2 does not simply ask stu¬ 
dents whether they can recognize the definition of 
person-who statistics, but rather it requires them to 
understand the underlying concept. Moreover, it 
calls on their analytical skills by requiring them to 
attribute to the author a particular point of view and 
to apply that point of view to a situation that they 
have not previously encountered. Thus, Question 2 
goes far beyond simply testing students’ ability to re¬ 
member information that they have encountered in 
the course. 

A third strategy that I encourage teachers to 
consider involves the use of interpretive exercises, 
which are sometimes referred to as item sets (Hala- 
dyna, 1992). In an interpretive exercise, students are 
first presented with novel introductory material, such 
as a reading, scenario, data set, chart, or map. They 
must then use the skills that they have developed in 
the course to answer a series of multiple-choice ques¬ 
tions that relate to the introductory material. The 
following is the introductory information from one 
of the interpretive exercises that I have used when 
teaching introductory statistics: 

Madame Clousseau claims to be a psychic 
- that is, she claims to be able to predict 
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future events with a level of accuracy that 
is substantially better than chance. To ex¬ 
amine her claim, Professor Jones brings 
her into his laboratory and tests her under 
carefully controlled conditions. He tosses a 
standard, fair coin 300 times and has Ma¬ 
dame Clousseau predict what the outcome 
will be for each toss. He finds that she cor¬ 
rectly predicts the outcome for 157 of the 
tosses. When he carries out the statistical 
test to analyze the results, Professor Jones 
lets alpha equal 0.05 and he uses a two- 
tailed test. 

This research scenario, which of course the students 
have never seen before the test, is followed by a series 
of four-alternative multiple-choice questions, such 
as “Which statistical test should Professor Jones use 
to analyze the data?” and “What would be the criti¬ 
cal value(s) for the test statistic?” Note that students 
can correctly answer these multiple-choice questions 
only by referring to the introductory information. 
Moreover, answering correctly is not simply an exer¬ 
cise in fact-finding. Rather, students must be able to 
understand and interpret what they have read, and 
they must then call on the statistical skills that they 
have developed in the course to answer the questions 
correctly. Thus, this interpretive exercise goes well be¬ 
yond testing whether students remember facts and 
concepts that they have encountered in the course, 
and instead calls on a variety of higher-level cognitive 
processes. 

Generally speaking, interpretive exercises are 
quite robust because they can be used for testing in 
a wide variety of disciplines. For instance, a teacher 
of poetry might have students read a poem for the 
first time and then answer multiple-choice questions 
dealing with its form, style, and content. Or a music 
teacher might have students listen to a recording for 
the first time and answer multiple-choice questions 
that involve the identification of vocal and instru¬ 
mental components as well as elements of form. 

I have presented several strategies that can 
help teachers to generate multiple-choice questions 
that assess higher-level cognitive processes. I am fully 
aware, of course, that multiple-choice questions have 


clear limitations. For instance, they are not at all use¬ 
ful for determining whether students can structure 
an argument, develop novel ideas, or invent some¬ 
thing new. Such abilities are best assessed using other 
techniques, such as essays, term papers, and projects. 
Nevertheless, multiple-choice questions can be used 
on tests to assess higher-level cognitive processes in 
ways that many teachers seem to overlook. Moreover, 
some teachers who have underestimated the value of 
multiple-choice questions may find that they can play 
quite a useful role in the assessment process. This is 
especially true for larger classes in which the grading 
of written answers can involve considerable time and 
effort. 
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